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ABSTRACT 

In this paper we consider non convex control problems of stochastic differential equa- 
tions driven by relaxed controls. We present existence of optimal controls and then 
develop necessary conditions of optimality. We cover both continuous diffusion and 
Jump processes. 
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1 Introduction 

The basic idea of the deterministic minimum principle introduced by Pontryagin and 
his colleagues in the 1950's is to derive a set of necessary and sufficient conditions that 
must be satisfied by any control which yields an optimal cost or pay-off. It consists 
of a system of forward-backward differential equations (e.g., state and adjoint equa- 
tions) and the extremum of a Hamiltonian functional. Since then the theory has been 
extensively developed in many directions, such as, optimal control theory for finite 
dimensional deterministic systems with regular controls [TJ [2], where one can find 
a broad and deep generalization of the classical Pontryagin minimum (equivalently 
maximum) principle for deterministic systems. The minimum principle is also ex- 
tended to infinite dimensional systems, see [31 HI El [6] and the references cited therein. 



The stochastic minimum principle is another important extension of the Pon- 
tryagin minimum principle for systems subject to probabilistic randomness. In the 
stochastic case, there are basically different approaches based on the assumptions 
employed to derive the stochastic minimum principle. Specifically, [7] utilizes spike 
variations and Neustadt's variational principle, [8] utilizes Girsanov's measure trans- 
formation for non degenerate controlled diffusion processes, while (9] utilizes the mar- 
tingale representation to derive the adjoint equation. The martingale representation 
approach is further developed in [TQl ITT] . Further results utilizing the martingale 
representation approach are established in [12] for control dependent diffusion pro- 
cesses utilizing second-order variations leading to a minimum principle which differs 
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from the deterministic case in the sense that the effect of control dependent diffusion 
terms are fully explored. Subsequent extensions are given in [13] for stochastic sys- 
tems with random coefficients, in [T3] utilizing stochastic flows to derive results similar 
to [12], and in [15] establishing relationships between stochastic minimum principle 
and dynamic programming. The martingale approach to stochastic minimum prin- 
ciple sparked the interest in studying backward and forward stochastic differential 
equations. An excellent account on the stochastic minimum principle is found in [TB] 
which also includes an anthology of references. Extensions of the stochastic maxi- 
mum principle for relaxed controls using the topology of weak convergence are found 
in [T71 HH [TH [201 [21], where relations to strict controls are also investigated. 
Recent developments and extensions are found in [221 1231 [21] and references therein. 
The area of mathematical finance, specifically portfolio optimization, has utilized the 
stochastic minimum principle extensively to derive optimal strategies. 

In general, the stochastic minimum principle is specific to the information struc- 
tures available to the control. Specifically, in applications of control theory, there are 
many problems in physical sciences and engineering, where systems are modeled by 
stochastic differential equations driven by controls which are also stochastic processes 
with specific information structure, such as, full information or partial information. 
Mathematically, information structures are modeled via the minimal sigma algebra 
generated by the available information process, and it is this process that the con- 
troller uses to generate control actions. For full-information problems in which the 
information structure is Markovian, one often employs Bellman's principle of opti- 
mally to construct, what is known as, the HJB (Hamilton- Jacob-Bellman) equation, 
a nonlinear PDE defined on the state space of the system under investigation. This 
equation describes the evolution of the value function which is used to construct the 
state feedback control law provided this function is at least once differentiable with 
respect to the state variable. This however requires solving the HJB equation which 
may have a viscosity solution but not sufficiently smooth [16]. For non-Markovian 
controlled diffusion systems with general information structures the HJB equation 
does not apply. For information structures which correspond to full information or 
partial information the stochastic minimum principle is often employed [TT1 1251 [6], 
although the partial information case is mathematically more demanding. However, 
this line of research is feasible provided existence of optimal controls is guaranteed. 
For non convex control problems, it is well known that the problem may have no opti- 
mal solution if the admissible controls are merely measurable functions with values in 
the set U which is non convex. Nevertheless, this problem can be partially overcome 
by introducing the relaxed controls and then approximating the relaxed controls by 
the standard regular controls. 

In this article we consider stochastic control systems with information structures 
corresponding to full information and partial information, which are driven by re- 
laxed controls. Specifically, controls which are conditional probability distributions, 
measurable with respect to full or partial information. We treat stochastic differential 
equations driven by both Brownian motion and Levy process or Poisson jump process. 
We show existence of optimal policies among the class of relaxed controls under gen- 
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eral conditions, with respect to an established topology of weak* convergence. Then 
we proceed with the derivation of stochastic minimum principle, for both the full 
information and the partial information cases. The Hamiltonian system of equations 
is derived in a systematic manner utilizing the semi martingale representation the- 
orem and the Riesz representation theorem, leading very naturally to the existence 
of the adjoint processes satisfying a Backward stochastic differential equation in an 
appropriate space. We also discuss the realizability of relaxed controls by regular con- 
trols using the Krein-Millman theorem. The methodology we consider is applied to 
stochastic differential equations driven by both Brownian motion and Poisson jump 
process. The basic procedure follows the one introduced in [5], [2] for deterministic 
systems, augmented by the martingale representation approach to stochastic control. 
The material presented for full information compliment the previous work on relaxed 
controls found in [171 HU [191 EH IZE] , where the authors utilize alternative methods 
to derive related results. 

The rest of the paper is organized as follows. In section 2 we present some typical 
notations and formulate the optimal control problem considered in this paper. In 
section 3, we consider the question of existence of optimal relaxed controls. Section 
4 contains an interesting fundamental result characterizing semi martingales. Here 
we construct a Hilbert space characterizing the space of semi martingales (starting 
from zero). This is used later in the development of necessary conditions. Section 5 
is devoted to the development of necessary conditions of optimality. In section 6 we 
extend the previous results to cover stochastic systems driven by jump processes. In 
section 7, we specialize to regular controls and obtain the usual necessary conditions of 
optimality. In section 8 we address the question of realizability of relaxed controls by 
regular controls. The paper is concluded with some comments on possible extensions 
of our results. 

2 Formulation of Stochastic Relaxed Control Problem 

In this section we introduce the mathematical model for the stochastic control system 
and the pay-off functional as a measure of performance. The distinction between full 
and partial information structures are also presented. 

Let (ft, J 7 , J-i>o, P) denote a complete filtered probability space where {Ft,t > 0} 
is an increasing family of subsigma algebras of the cr-algebra J 7 . For any random 
variable z, £{z) = L z{uj)P(du) denotes the expected value (average) of the random 
variable z. Let {W(t),t > 0} denote the R m - valued standard Brownian motion with 
P{W(0) = 0} = 1 defined on the filtered probability space (f2, J 7 , J-"t>o, P). Let 
Qt C Tt denote a family of sub-sigma algebras of the cr-algebra Tt-, t > 0. 

Let / = [0, T] be any finite interval, U any closed bounded subset of R d and Ai(U) 
the space of regular bounded signed Borel measures on B(U), the Borel subsets of 
U and A4i(U) C Ai(U) the space of regular probability measures. Controls based 
on partial information (respectively full information) will be described through the 
topological dual of the Banach space L1(I,C(U)), the Lx-space of Q t (respectively 
Ft) adapted C{U) valued functions. The dual of this space is given by L^JJ, M.(U)) 
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which, for partial information, consists of weak star measurable Q t adapted Ai(U) 
valued functions (signed measures), while for full information it consists of Tt adapted 
functions defined similarly. For controls based on partial information (respectively 
full information) we are interested in the subspace L^I , Aii(U)) C L^I, M.(U)) 
of probability measure valued Q t (respectively J^) adapted functions. Let U a d = 
L^I , A4i(U)) denote the class of admissible controls, called the relaxed controls, 
where the distinction between full information and partial information is only specified 
in terms of the cx-algebras J-" t and Qt, respectively. 

We consider the following stochastic system in R n governed by the Ito differential 
equation which is driven by relaxed control, 

dx(t) = ( / b{t, x(t),£)u t (d£)) dt+( a{t, x{t),^)u t {d^)) dW(t), x(0) = x , t G /, 

where b : I x R n x U — > R n denotes the drift and a : / x R n x U — > C(R m , R n ) 
the diffusion parameters. For simplicity of notation we prefer to write the above 
controlled dynamic system in the form 

dx(t) = b(t, x(t),u t ) dt + a(t, x(t),u t ) dW(t),x(0) = x , t G /, (1) 

for any u G U a d- The cost functional is given by 

J(u) = £ ^J q i(t,x(t),Ut)dt + $(x(T))y 

The problem is to find a control u° G U ac i such that J{u°) < J{u) for all u G U a d- 
We consider the question of existence of optimal controls and characterization of 
such controls in the form of necessary conditions of optimality (Pontryagin minimum 
principle). For necessary conditions of optimality we follow the procedure developed 
in [2], pp. 271-293 which we extend from deterministic to stochastic systems. 

3 Existence of Optimal Relaxed Controls 

Consider the system ([1]) with b and a denoting the infinitesimal generators represent- 
ing the drift and diffusion given by the Borel measurable maps: 

b:IxR n xR d — > R n ,a : I x R n x R d — ► £(R m , R n ). 

We assume that they satisfy the following basic properties: there exists a K G 
L^"(/)(nonnegative functions belonging to L 2 (/)), such that 

(Al) : \b(t,x,Z)-b(t,y,i)\ Rn <K(t)\x-y\ Rn uniformly in £ G U 

(A2) : \b{t,x,i)\ Rn < K(t)(l + \x\ Rn ) uniformly in f G U 

(A3) : |a(t,x,0 - a(t, y, 0|£(jp»,jp») < K{t)\x-y\ R * uniformly in f G U 

(A4) : \a{t, x, C)\c(R m ,R n ) < K{t){l + \x\ R n) uniformly in £ G U 

(A5) : b(t,x,-),a(t continuous in £ G U uniformly in t G [0, T], x G R n . 
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For admissible controls, we choose the set of relaxed controls given by U a d = 
L^ (7, M.i(U)) which are stochastic processes, adapted to a given sigma algebra (to be 
specified later), and taking values in the space of probability measures M\(U). This is 
endowed with the weak star topology also called vague topology. A sequence u n G U a d 
is said to converge vaguely to u°, written u n — > u°, iff for every ip G L1(I, C(U)) 

£ I <p(t,£)ut(d£)dt ->• £ / (p(t,£)u°(d£)dt as n ->■ oo. 

JlxU JlxU 

With respect to this vague (weak star) topology, U a d is compact and from here on we 
assume that U a d has been endowed with this vague topology. 

Let -6^(7, L 2 (fi, R n )) denote the space of J-j-adapted R n valued second order 
random processes endowed with the norm topology || • || given by 

|| x f = sup{£ \x{t)\ 2 Rn ,t G /}. 

With this preparation, we can now present the following lemma proving existence 
of solutions and their continuous dependence on controls. 

Lemma 3.1 Consider the controlled stochastic differential equation ([T]) and suppose 
the assumptions (A1)-(A5) hold. Then for any J-o- m easurable initial state xq having 
finite second moment, and any u G U a d, the system flTJ has a unique solution x G 
-8^(7, Li2(Q, R n )) having continuous modification. In other words, x G C(I,R n ) P- 
a.s. Further, the solution is continuously dependent on the control in the sense that 
as u n — > u° in U ad , the corresponding solutions x n — ^ x° in B^(I, L 2 (£l, R n ). 

Proof. The proof for the first part of the lemma is classical and hence we present 
only an outline. It is based on the Banach fixed point theorem applied to the operator 
F on the Banach space -8^,(1, L 2 (fl, R n )) where 

(Fx)(t)=x + [ b{s,x{s),u s ) ds + / a(s, x(s), u s ) dW(s), t G I = [0, T\. (2) 
Jo Jo 

Under the assumptions (A1)-(A4), it is easy to verify using classical martingale in- 
equality that F : -B^(J, L 2 (0, R n )) to itself. Then using the metric d given by d = dr 
where 

d 2 t (x,y) = sup{£|x(s) -y(s)\ 2 Rn ,0 < s < t} 

for t G I, one can verify that the n — th iterate of F denoted by F n = FoF ■ ■ ■ oF (n 
times) is a contraction. Then by Banach fixed point theorem F n has a unique fixed 
point in B^I, L 2 (f2, -R n )) and hence F itself has one and the same fixed point [1]. The 
continuity of the sample paths however follows from classical Borel-Canteli lemma. 
Now consider the second part asserting the continuity of the control to solution map 
u — > x. For this one proceeds as follows. Suppose the assumption (A5) holds and let 
{u n ,u°} be any sequence of controls from U a d and {x n ,x°} denote the corresponding 
sequence of solutions of the system (JTJ). Let u n u°. We must show that x n — x° 
in B^I, L 2 (ft, R n )). We present only a hint. Using the definition of solution, it is 
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easy to verify that 

x n {t) -x°{t) = f [b{s,x n {s),u n s ) -b{s,x°{s),u^)}ds 
Jo 

+ / [a(s,x n (s),u:) - a(s,x°(s),u:)]dW(s) + e hn (t) + e 2tn (t),t e I 
Jo 

(3) 

where 

ei,n{t) = [ t [b(s,x°(s),u:) -b(s,x°(s),u° s )]ds 
Jo 

e 2 ,„(t)= / [a(s,x°(s),u:) - a(s,x (s),u° s )}dW(s). 
Jo 

Using the standard martingale inequality it follows from this that there exist constants 
Ci, C 2 > such that 

E\x n (t)-x°(t)\ 2 <d f K 2 (s)E\x n (s)-x°(s)\ 2 + C 2 (£\e ltn \ 2 + £\e 2>n \ 2 ). (4) 

Jo 

Clearly, 

£\e lin \ 2 <T£ [ \b(s,x (s),u n s )-b(s,x (s),u° s )\ 2 Rn ds 
Jo 

and 

£|e 2 ,n| 2 < 4£ / \a(s,x°(s),u™) - a(s,x°(s),u s )\ 2 ciR m jRn) ds. 
Jo 

Now note that by virtue of vague convergence of u n to u°, the integrands of the above 
inequalities converge to zero for almost all s £ I, P-a.s and it follows from (A2) and 
(A4) that they are dominated by integrable functions. So by Lebesgue dominated 
convergence theorem the integrals {ei jn , &2,n\ converge to zero uniformly on I. The 
assertion then follows from Gronwall inequality, applied to the inequality (T4J). This 
completes the outline. • 

Optimal Control Problem. Consider the controlled system (pQ) and the cost func- 
tional given by 

J{u)=£{f £{t,x{t),Ut)dt + $(x{T))} (5) 
Jo 

where i and $ are suitable functions which are measures of mismatch between the 
desired flow and the flow that results from the choice of the control u. The problem, as 
stated in section 2, is to find a control from the class of admissible (relaxed) controls 
U a d that minimizes the functional (jSJ). We present the following existence result. 

Theorem 3.2 Consider the control problem as stated above. Suppose the assump- 
tions of Lemma 3.1 hold, and further suppose £ : J x R n x U — > (— oo, +oo] and 
$ : R n — > (— oo, +oo] are Borel measurable maps satisfying the following conditions: 
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(al): x — > £(t,x,£) is continuous on R n for each t 6 I, uniformly with respect to 
(a2): 3 he L~{{I) such that \£(t } x,g)\ < h(t)(l + \x\ 2 Rn ) 

(a3): x — > $(x) is lower semi continuous on R n and 3 co,ci > such that 
< c + Ci\x\ 2 Rn . 

Then, there exists an optimal control u G U a d & t which J attains its minimum. 

Proof. Since U a & is compact in the vague topology, it suffices to prove that J is 
lower semi continuous with respect to this topology. Suppose u 11 u° in U a d and 
let {x n ,x°} C B^I , L 2 (fl, R n )) denote the solutions of equation (jTJ) corresponding 
to the sequence of controls {«", «"} C U a d- Then by Lemma 3.1, along a subsequence 
if necessary, x n — x° in B 00 (I,Li2{VL,R n )). First note that, in view of the strong 
convergence, along a subsequence if necessary, x n (T) — > x°(T) P-a.s. Thus it follows 
from assumption (a3) and Fatou's Lemma that 

£{$(x°(T))} < liminf £{$(x n (T))}. (6) 

n 

Considering the running cost, it is easy to see that 

£ J^{t,x°{t),u%) dt = £ J t{t,x°{t),u°t -<) dt 

+£ J(e(t,x°(t),i4) -£(t,x n (t),u?)) dt + £ J^£(t,x n (t),u?) dt. (7) 

By virtue of vague convergence of u n to u°, it is evident that for every e > there 
exists an integer n l e sufficiently large, such that the absolute value of the first term 
on the right hand side of equation (j7|) is less than e/2 for all n > By virtue of 
assumption (al)-(a2), in particular the continuity of £ in x uniformly in U, it is easy 
to verify that there exists an integer ri2 t£ such that for all n > n 2te , the absolute value 
of the second term on the right hand side is less than e/2. By combining these facts 
we obtain the following inequality 

£ f e(t, x°(t), u°) dt<s + Je(t, x n (t), u n t )dt 

for all n > nx >e \J n^ jE . Since e > is otherwise arbitrary, it follows from the above 
inequality that 




Combining ([6]) and (jSJ) we arrive at the conclusion that J(u°) < liminf „ J(u n ) thereby 
proving lower semi continuity of J in the vague topology. Since U a d is compact in this 
vague topology, J attains its minimum on it. This proves the existence of an optimal 
control. • 

Note that the existence is proved under general conditions, irrespectively of 
whether the information structure to the control is full or partial. 



Stochastic Minimum Principle for Relaxed Controls 



8 



4 Construction of a Hilbert Space of Semi Martingales 

In the preceding section we have presented a result on existence of optimal controls. 
In the following section we consider the problem of characterizing optimal controls in 
the form of necessary conditions of optimality. For this we shall utilize martingale ap- 
proach hence we need to consider certain fundamental properties of semi martingales. 
These properties are studied in this section. Before we consider such properties, we 
wish to provide the technical reasons for their study. Consider the system ([1]) with 
the cost functional (JSJ and the admissible controls U a d = L^I, Mx(U)) as described 
above. Recall that these are either Tt or ^-adapted probability measure valued ran- 
dom processes, depending on whether the information structure used to construct the 
controls is full or partial. For the necessary conditions of optimality we need stronger 
regularity properties for the drift and diffusion parameters {b, a} as well as the cost 
integrands {£,$}. They are presented as follows: 

(NCI): The triple {b, a, £} are measurable in t G I, and the quadruple {b, a, £, $} 
are once continuously different iable with respect to the state variable x G R n . The 
first spatial derivatives of {b, a} are bounded uniformly on / x R n x U. 

Considering the Gateaux derivative of a with respect to the state variable at the 
point (t, z,v) G I x R n x Mi(U) in the direction rj G R n we have 

lim(l/e) (a(t, z + er], v) - a(t, z, v)) = aJt, z, u; rj). 

e-s>0 

Note that rj — > a x (t, z, v\ rj) is linear and it follows from the assumption (NCI) that 
there exists a finite positive number such that 

\(T x (t,z,v]7f)\c(am,Rn) < /3\v\r«- 

In order to present the necessary conditions of optimality we need the so called varia- 
tional equation. Suppose u° G U ac [ denote the optimal control and u G U a d any other 
control. Since U a d is convex, for any e G [0, 1], the control 

U 6 = U° + S(U - U°) G Uad- 

Let x £ ,x° G B^I , L 2 (fl, R n )) denote the solutions of the system equation ([I]) corre- 
sponding to the controls u e and u° respectively. Consider the limit 

y = lim(l/£:)(x e — x°). 
We have the following result characterizing the process y. 

Lemma 4.1 The process y is an element of the Banach space -B^(J, L 2 (fl, R n )) and 
it is the unique solution of the variational SDE 

dy(t) = b x (t,x°(t),u°) y(t) dt + a x (t,x°(t),u° t ;y(t)) dW(t) 

+ b{t, x°(t),u t - u° t ) dt + a{t, x°(t),u t - u°) dW(t),(9) 

2/(0) = 0, 



having a continuous modification. 
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Proof. This is a linear SDE and so one can have a closed form solution. Indeed, 
considering the homogenous part given by 

dz(t) = b x (t, x°(t),u°) z(t) dt + a x (t, x°(t), u° t ; z{t)) dW(t), z(s) = C, < s < t < oo, 

it follows from the assumption (NCI) and Lemma 3.1 that it has a unique solution z 
given by 

z(t) = V(t,s)(,t>s, 

where \I/(t, s), < s < t < oo is the random (J-" 4 measurable) transition operator 
for the homogenous system. Since the spatial derivatives of b and a are uniformly 
bounded, the transition operator $?(t, s), < s < t < T is uniformly P almost surely 
bounded (with values in the space of nxn matrices) . Considering the non homogenous 
system ([9]), the solution is then given by 

y(t) = [ *(t,s)dr)(s) (10) 
Jo 

where 7] is the semi martingale given by 

drj(t) = b(t, x°(t), u t - u°) dt + a{t, x°(t),u t - u° t ) dW(t), rj(0) = 0. (11) 

Note that 77 is a continuous square integrable Tt semi martingale. This proves the 
existence, uniqueness and regularity property of the solutions of system ([9]). This 
is one approach. An alternate approach is the same as that of Lemma 3.1. Here 
one notes that the drift and the diffusion terms of equation satisfy the basic 
assumptions of Lemma 3.1. So the existence of a solution follows from the Banach 
fixed point theorem as in lemma 3.1. The fact that it has continuous modification 
follows directly from the representation ([TO]) and the continuity of the semi martingale 
77. • 

Later in the sequel we need certain important and interesting properties of semi 
martingales. Let L^I^R 11 ) C L 2 (I x Q, R n ) denote the space of J-^-adapted random 
processes {v(t),t G /} such that 

£ J \v{i)\ 2 Rn dt < 00. 

Similarly, let L|(i ', C(R m , R n )) C L 2 (I x f2, C(R n \ R n )) denote the space of T t - 
adapted n x m matrix valued random processes {S(t),t G /} such that 

£ J \Z{t)\ 2 c{Rm , Rn) dt = £ J tr(E*(t)E(t))dt < 00. 

Since / is a finite interval , it is clear that B^(I, L 2 (tt, R n )) C L%(I, R n ). 

Definition 4.2 An i? n -valued random process {m(t),t G /} is said to be a square 
integrable continuous J-j-semi martingale iff it is representable in the form 

m(t)=m(0)+ [ v{s)ds+ [ E(s)dW{s),t G /, (12) 
Jo Jo 
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for some v G L%(I,R n ) and £ G L a 2 (I , C(R m , R n )) and for some iT-valued T mea- 
surable random variable m(0) having finite second moment. 

We introduce the following class of J^-semi martingales: 
SMl = jm : m(t) = v(s)ds + J Z(s)dW(s),t G /, 

for v G L a 2 {I, R n ) and £ G L£(J, £(iT\ R n )) \. (13) 



Now we present a fundamental result which has the potential of many other ap- 
plications. 

Theorem 4.3 The class SM 2 is a real linear vector space and it is a Hilbert space 
with respect to the norm topology || m \\sm% arising from 



m \\ 2 sm 



2= £ \v{t)\ 2 Rn dt + £ Jtr(Z*(t)X(t))dt. 



Further, the space SMl is isometrically isomorphic to L 2 (I, R n ) x L 2 (I, C(R m , R n )), 
written as SM 2 - Lti 1 , Rn ) x £( Rm , Rn ))- 
Proof Note that each m G SMl corresponds to a pair 

(v,Z)eL a 2 (l,R n )xL a 2 (l,£(R m ,R n ))- 

We may call the pair (v, S) the infinitesimal generator (or simply the intensity) of the 
semi martingale m. Let mi G SMl corresponding to the intensity process (i>i, Si) G 
L 2 (I, R n ) x L 2 (I, C(R m , R n )) and m 2 G SMl corresponding to the intensity process 
(v 2 , S 2 ) G L%(I, R n ) x L%(I, £{R m , R n )) respectively. Clearly Vl + v 2 G L%(I, R n ) and 
£i + £ 2 G ^(i", £( Rm , R n ))- Hence m = m 1 + m 2 , with intensity process (i>i+i>2, Si + 
S 2 ), is an element of SMl- For any real number a and any m G SMl with intensity 
process (u, £) G L^{I,R n ) x L^(I,C{R m ,R n ) we have am G <S.Mo with intensity 
process (m;, aE) G L 2 (I,R n ) x L 2 (I , C(R m , R n ) . Thus <S.Mo is a linear vector space. 
We now furnish this with a scalar product and norm topology. Let m 1 ,m 2 G SMl 
with the intensity pairs (vi, £1), (i>2, £2) respectively and define 



(mi,m 2 ) 5M 2 =£ J( Vl (t),v 2 (t))dt + £ J^tri^lit^t^dt. 



(14) 



The reader can easily verify that this gives a scalar product. Clearly taking m 2 = mi 
we have the norm square of m x given by 

II m i Wsm?= ( m ^ m i)sM 2 = ^ JjMt)\ R ^dt + £ J^i(t)\ 2 c{Rm Rn) dt. (15) 

It is easy to verify that the above expression defines a norm (modulo the null space). 
Thus SMl is a scalar product space. To show that it is a Hilbert space, it suffices to 
verify that it is complete. Let {m n } C SMl t> e a Cauchy sequence corresponding to 
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the sequence of intensity pairs {(v n , E n )} C L%{I, R n ) x L%(I, C(R m , R n )). Let p > 1 
and consider the expression 

II m n+p ~ m n WsMq = J \ v n+p(t) — V n (t)\ R „dt -\- S J |E n _).p(t) — ^ n \c(R m ,R n )^ 

Since {m n } is a Cauchy sequence, lim n _s. 00 || Tn n j r p — ?n n Hg^s — for every p ^ 1 
and hence {{v ni S„)} is a Cauchy sequence in L%(I, R n ) x /^(-^ £(i? m , -R n ))- But the 
later spaces are Hilbert and hence there exists a unique pair (v , E ) G L%(I,R n ) x 
/^(i, C(R m , R n )) to which (v n , E n ) converges in norm (along a subsequence if neces- 
sary). Define the process mo by 



m (t)= / v (s)ds+ / E (s)dW(s),t G /. 
Jo Jo 

Clearly this is a semi martingale belonging to SJ\Aq and it is the unique limit of the 
sequence of semi martingales {m n }. This proves that S/\4q is complete and hence a 
Hilbert space. Now we claim that for every m G SJ\4q there exists a unique pair 
{v, E) G (7, iT) x £,«(/, £{R m , R n )) such that 



m(t) = / v(s)ds+ / E(s)dW(s),t G J. 
Jo Jo 

Suppose this is false and there exists another pair (v%, Ei) G -^2(^5 # n ) xL§(7, £(R m , R n )) 
giving the same semi martingale m. This means that 

0= f (v(s) - Vl {s))ds + [ (E(s)-E 1 (s))dW(s),teI, 
Jo Jo 

which is the same as 

t r-t 

(v(s)-v 1 (s))ds= / (Ei(s) - E(s))dW(s), tel. 
Jo 

But this is impossible since a martingale can never equal a function of bounded 
variation. Hence V\ = v and Ei = E. Thus to every m G SMq there corresponds 
a unique pair (v, E) G L^I, -R n ) x L?,(I , C(R m , R n )) and conversely. The isometry 
follows from the expression flT^. Hence SyVlg ^ L a 2 (I,R n ) x L%(I , £(R m , R n )) ■ This 
completes the proof. • 



5 Necessary Conditions of Optimality 

Now we are prepared to develop the necessary conditions of optimality. The theory 
of relaxed controls is found to be a powerful technique for developing necessary con- 
ditions of optimality for deterministic systems [2], Theorem 8.3.5. Here we use the 
same technique for systems governed by stochastic differential equations driven by 
relaxed controls. 

Below, we provide the main theorem. Later we use this result to derive a simplified 
minimum principle for both full as well as partial information. 
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Theorem 5.1 Consider the system (pQ) and the cost functional (jSJ). An element 
u° G Uad, with the corresponding solution x° G B^(I , L 2 (fl, R n )) to be optimal, it is 
necessary that there exists a semi martingale m° G SJ\4q with the intensity process 
(ip,Q) G L%(I,R n ) x L^(I,C(R m ,R n )) such that the following inequality and the 
equations (SDE) hold: 



(!):£/ {(6(t,x (t),M 4 )^W)+^(g*(t) ( T(t,x (t),^))+^(t,x (t),^)}rft 



> £ / { (b(t, x°(t),u°)^(t)) + tr(Q*(t)a(t, x°(t),u°)) + £(t, x°(t),u° t )}dt 
Jo 

for all u G U a d- (16) 



(2) : dx°(t) = b(t,x (t),u° t )dt + a(t,x (t),u° t ))dW(t) 

x°(0) = x (17) 



(3) : -dif>(t) = b* x {t, x (t),u t )ip(t)dt + V Q (t)dt + £ x {t, x°(t),u°)dt - Q(t)dW(t) 
V>(T) = $,(x°(T)) (18) 

where V Q G L a 2 (I,R n ) is given by (Vg(*),C) = tr{Q*{t)a x {t, x°(t), u° t ; ()), te L 

Proof Suppose u° G U a d is the optimal control and u G U a d any other control. 
Since U a d is convex, for any e G [0, 1], the control u e = u° + e(« — u°) G W a d- Let 
x e ,x° G B^(I , L 2 (£l, R n )) denote the (strong) solutions of the system equation (pQ) 
corresponding to the controls u e and u° respectively. Since u° is optimal it is clear 
that 

J{u £ ) - J{u°) > (19) 

for all e G [0, 1] and for all u G U a d- Let dJ(u°, u — u°) denote the Gateaux differential 
of J at u° in the direction u — u°. Dividing the expression (TT9~j) by e and letting e \. 
it is easy to verify that 

dJ{u°,u-u°) = L(y)+£ I £{t,x°(t),u t - u° t )dt > 0, M ueU ad (20) 
where is given by the functional 

L(y) =£y\e x (t,x°(t),u°),y(t)) dt + ($ x (x°(T)), j/(T))J. (21) 

Since by Lemma 4.1, the process y G B^(I , L 2 (fl, R n )) and it is also continuous P- 
a.s it follows from assumption (a2) of Theorem 3.2 and the assumption (NCI), that 
y — > L(y) is a continuous linear functional. Further, by Lemma 4.1, 77 — > y is a 
continuous linear map from the Hilbert space SM.^ to the B-space B^I, L 2 (f2, R n )) 
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given by the expression (jTOj) . Thus the composition map rj — > y — > L(y) = 
L(rj) is a continuous linear functional on SM.Q. Then by virtue of the classical Riesz 
representation theorem for Hilbert spaces, there exists a semi martingale g G SAi^ 
with intensity (ip,Q) G L a 2 (I,R n ) x L a 2 (I , C(R m , R n )) such that 

L(y) = L(ri) = (g, V ) SM 2 = £ [ (<i[>(t), b(t, x°(t),u t - u°))dt 

Jo 

+ £ tr(Q*(t)a(t,x°(t),u t - u° t ))dt. (22) 
Jo 

Substituting the expression ( )22l into the expression (|20l) we obtain 

dJ(u°, u-u°) = £ [ (ip(t), b(t, x°(t), u t - u°))dt 
Jo 

+ £ [ tr{Q*{t)a{t,x°{t),u t -u° t )) 
Jo 

+ £ £{t, x°(t), u t - u°)dt > 0, Vug U ad . (23) 
Jo 

The necessary condition given by the expression ( |T6|) readily follows from this. Equa- 
tion (1TT1) is the system equation along the optimal control state pair (u°,x°), so 
nothing to prove. We prove that the pair (if), Q) is given by the solution of the ad- 
joint equation (|T8l) . Computing the Ito differential of the scalar product (y,ip) we 
have the general expression 

d{y{t),iP{t)) = {dy{t),iP(t)) + {y(t),diP{t))+ < dy(t),dil>(t) > (24) 

where the last bracket denotes the classical quadratic variation term. Integrating 
this over / = [0, T] and using the fact that y(0) = 0, it follows from the variational 
equation ([9]) that 



£{y{T)^{T)) = ^y\y(t), b^(t)dt + a* x (^(t))dW(t) + 

+ [ (b°,4>(t))dt+ [ {{a°)*iP{t),dW{t))\+£ I <dy{t),diP{t)>,{25) 
Jo Jo ) Jo 

where for convenience of notation we have used 

b x = b x (t,x°(t),u° t ), a x (0 = a x (t,x°(t),u°;0,£e R n , 
b° = b(t, x°(t), u t - u°), a° ee a(t, x°(t),u t - u°). 

Note that the stochastic integrals in (1251) equal zero and hence make no contribution. 
This follows from the facts that a* x {ip{t)) G L%(I, C(R m , R n )) and (a°)*ip G Lf (/, R m ) 
as seen later. So we can eliminate them giving the following expression 

£{y(T),iP{T)) = fjjT {y(t)X^{t)dt + diP(t))+ jf {b°^{t))dt 

+ £ [ < dy(t),dil)(t) > . (26) 
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Before we consider the quadratic variation term, let us recall that the Ito derivatives 
of the variation process y and the adjoint process if> are of the following form: 



dy(t) = bounded variation terms + a x (t, x°(t), u° t \ y(t))dW(t) 

+ a(t,x°(t),u t -u°)dW(t), 
dip(t) = bounded variation terms + Q(t)dW(t). 

Considering now the quadratic variation term it is easy to verify that 

£ f < dy(t),d*l>(t) >= £ f \tr(Q*(t)a x (y)) + tr(Q*(t)a°)}dt. (27) 
Jo Jo 

Clearly, the first term on the right hand side of the above expression is linear in y. 
Thus there exists a process Vq(t),t G /, given by the following expression 

(V Q (t),y(t)) = tr(Q*(t)a x (y)) = tr(Q*{t)a x (t,x°(t),u° t ;y(t))). (28) 

By assumption (NCI), a has uniformly bounded spatial first derivative and it follows 
from the semi martingale representation Theorem 4.3 that Q G L?,(I, C(R m , R n )) and 
hence V Q G L%(I,R n ). Substituting flU} into (J2ZD and then (J2ZD into (J2SD, we obtain 

£{y{T)^{T)) = 5 1 J (y(t), b*Mt)dt + V Q (t)dt - Q(t)dW(t) + #(t)) 

+ I (b°,ip{t))dt + tr{Q*(t)(T )dt 
Jo ) 

(29) 

By setting 

b* x (t, x°{t),u°)il){t)dt + V Q (t)dt - Q(t)dW(t) + dnj)(t) = -£ x (t, x°(t), u°)dt 
i>{T) = <f,(x°(T)), (30) 

it follows from (j29|) and the expression for the functional L given by (j2T|) that 
L(y) = £(y(T),i;(T)) + £ [ T (y(t)J x (t,x°(t),u°))dt 



o 



T 



£ {(b(t,x°(t),u t -u t )^{t))+tr{Q*{t)a{t,x {t),u t -u° t ))}dt. (31) 



This is precisely what was obtained by the semi martingale argument giving ( f2~2l ). 
Thus the pair (ip, Q) must satisfy the backward stochastic differential equation (13"01) 
which is precisely the adjoint equation given by ([TBI as stated. Since ip satisfies the 
stochastic differential equation and T is finite, it follows from the classical theory of Ito 
differential equations that if) is actually an element of -8^(1, L 2 (0, R n )) C L%(I, R n ). 
In other words, if) is more regular than predicted by semi martingale theory. Hence by 
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our assumption on a it is easy to verify that cr*(^>) G L^I, C(R m , R n )) and {cr°)*ip G 
L%(I, R m ) as stated before. Thus we have completed the proof. • 

Remark 5.2 Define the Hamiltonian 

H : I x R n x R n x C(R m , R n ) x M X {U) — > R 

by 

H(t, e, C, M, v) = (b(t, £, u),C) + tr(M*a(t, £, u)) + £(t, £, v). 

In terms of this Hamiltonian, the necessary conditions of optimality ( I16|) - (|18p can be 
written compactly as follows 



S / H(t, x°(t),iP(t), Q(t), u t )dt > S / H(t, x°(t)^(t), Q(t), u°)dt 

Jo Jo 
for all u G U ad , (32) 

where the triple {x°, ip, Q} is the unique solution of the following Hamiltonian system 

dx°(t) = H^(t, x°{t)^{t), Q(t), u°)dt + a(t, x (t),u t )dW(t), x°(0) = x , (33) 



#(t) = -H x (t,x°(t),i;(t),Q(t),u° t )dt + Q(t)dW(t), i/j(T) = ® x (x°(T)). (34) 

Note the similarity in appearance with the Pontryiagin minimum principle. In 
fact we recover the Pontryagin minimum principle for relaxed controls in [5J [2] by 
setting <t = 0. 

For controls based on full-information which are Tt adapted, and under the condi- 
tion that {J-'tjt G [0,T]} is the natural filtration generated by the Brownian motion 
{W(t),t G [0,T]}, augmented by all P— null sets in J 7 , given by the inequality ( TT6I) 
(or equivalently (132}) ) is equivalent to the following point wise almost sure inequality 
(the derivation is similar to that of Corollary 5.3): 

H(t,x°(t),ij(t),Q(t)^) > H(t,x°{t),i/;(t),Q(t),u t ), 
V> G Mi(U), a.e.t G [0,T], P - a.s. 

or equivalently, 

H(t,x°(t),m,Q(t),u° t )= min H(t,x°(t),^(t),Q(t),fi), 

fj.eMi(u) 

a.e.t G [0,T], P - a.s. 

subject to the Hamiltonian system fl3"3l - fl3~4"l) . 

For the partial information case, the point wise necessary conditions of optimality 
for controls are given in the next Corollary. 

Corollary 5.3 Suppose the assumptions of Theorem 5.1 hold and consider controls 
which are Qt adapted. Then the inequality (|T6|) (or equivalently (|32|) ) is equivalent to 
the following point wise almost sure inequality with respect to the cr-algebra Q t C Tt ■ 
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£{H(t,x°(t),m,Q(t)^)\Gt} > £{H(t,x°(t),il;(t),Q(t),u°)\g t } (35) 

for all /i G Aii(U), a.e.t G [0, T], P — a.s. subject to the Hamiltonian system (|33l)-( !34l) . 

Proof. Since the admissible controls are vaguely Q% measurable, we can rewrite the 
inequality (1321) in the following equivalent form, 

£ f £{H{t,x°(t),i>{t),Q{t),u t )\g t } dt 
Jo 

>£ [ £{H(t,x°(t),iP(t),Q(t),u°)\g t } dt. (36) 
Jo 

Let t G (0,T), oj G Q and e > and consider the sets I £ = [t,t + e] G I and 
fl £ (G Q) G Qt containing u such that \I £ \ — > and P(Q £ ) — > as e — > 0. For any 
subsigma algebra Q <Z J 7 , let Pg denote the restriction of the probability measure P 
on to the a-algebra Q. For any (vaguely) (^-measurable v g A^i(C/), construct the 
control 

v for (t, w) G I £ x fi £ 
otherwise. 



'U, 



Clearly, it follows from the above construction that u G U a d- Using this control in 
(I35p we obtain the following inequality 

£{H(t,x°(t),iP(t),Q(t),u)\g t } dt> ! £{H(t,x (t),iP(t),Q(t),u°)\g t } dt. 



(37) 



Letting \I £ \ denote the Lebesgue measure of the set I £ and dividing the above expres- 
sion by the product measure P(Q £ )\I £ \ and letting e — > we arrive at the following 
in equality, 

£{H(t, x°{t), Q(t), v)\Gt} > £{H(t, x°(t),1>{t), QV), 0\Qt} 

which holds for almost all t G / and Pg t almost all a; G fl Thus we have completed 
the proof. • 

Remark 5.4 Define 

9t {Z) = £{H(t,x°(t),i>(t),Q(t),®\g t },te I,£eU. 

The reader can easily verify from the basic assumptions on the parameters {b, a, £, $} 
that the random process g is an element of L®(I,C(U)) and that it is adapted to 
the cr-algebra Q t . Clearly, the necessary condition given by the inequality (|35|) can be 
written as follows 



Ju 
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and this must hold for all A^i(C/)-valued (^-adapted (vaguely measurable) random 
variables /i. Define 

a*(aO = / g t (OKdO- 

Ju 

This is a £?t- measurable continuous linear functional on ftA\(U). Since the later space 
is vaguely compact, it attains its minimum on J\4\(U) and from the above inequality 
it follows that u° t is one such element. Because the functional At is not strictly convex 
there may be multiplicities of minima M°(t). It is easy to verify that the set 

M°(t) = {/i e Mi(U) : n&Gt- measurable and A t (fj) = A t (u°)} 

is convex and a vaguely (weak star) closed subset of J\4i(U) and hence vaguely com- 
pact. Thus t — > M°{t) is a measurable multi function with convex compact values in 
M.x(U). By our assumption U is compact and hence M.\(U) is a compact Polish space 
and hence a compact Souslin space. Thus it follows from the well known Yankov- 
Von Neumann- Auman selection theorem [[26J , Theorem 2.14, pl58] that the multi 
function t — > M°(t) has a Q t measurable selection. Hence we have a Q t measurable 
optimal relaxed control. 

6 Extension to Jump Processes 

The necessary conditions of optimality given in the previous section can be easily 
extended to control problems involving stochastic differential equations driven both 
by Brownian motion and Levy process or Poisson jump process. Let Z = R n \ {0} 
and B(Z) the Borel algebra of subsets of the set Z. Let p(dv x dt) denote the Poisson 
counting measure on B(Z) x a {I). Physical interpretation of this measure is simple. 
For each V e B{Z) and any interval A e cr{I), p{T x A) gives the number of jumps 
over the interval A of sizes confined in T. This is a Poisson random variable with 
mean Sp(T x A) = 7r(r)A(A) where A is the Lebesgue measure on the real line and 7r 
is the Levy measure on Z. Here tt is a countably additive bounded positive measure. 
The compensated Poisson random measure is given by 

q(dv x dt) = p(dv x dt) — 7t(dv)dt. 

There is no loss of generality considering the compensated Poission random measure 
in modeling SDE. As usual, we assume that all the random processes considered in 
this paper are based on the filtered probability space (fi, J 7 , Tt>o, P) where {J- t , t > 0} 
is an increasing family of subsigma algebras of cr-algebra J 7 and that they are right 
continuous with left limits. A controlled stochastic differential equation driven both 
by Brownian motion and the compensated jump process described above is given by 
the following stochastic differential equation 

dx(t) = b(t, x(t),u t )dt + a(t, x(t),u t )dW(t) + / C(t, x(t),v, u t )q(dv x dt),t E I (38) 

Jz 

for x(0) = Xq. Throughout the rest of the paper it is assumed without any further 
notice that {x ,W,q} are independent random elements. Again our controls are 
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relaxed controls which, for the partial information case, are weakly Q t adapted Ai\(U) 
valued random processes denote by U a d- The cost functional is given by 

J{u) = 8{ [ £(t,x(t),u t )dt + <$>(x(T))\. (39) 



Objective is to find a control from the admissible set U a d at which the functional (1391) 
attains its minimum. The method of proof of the necessary conditions of optimality 
for this model is no different from the one given for the continuous case. Hence we 
present the results without repeating the detailed proof. 

For the problem involving jump process, we introduce the following Hilbert space 
of discontinuous square integrable semi martingales denoted by VSAA^ and this is 
given by 

VSMl = \ m ■ m (t) = [ v(s)ds + f Q(s)dW(s) + [ [ <p{v,t)q(dv x dt) 
I Jo Jo Jo J z 

v G L a 2 (I,R n ),Qe L a 2 (I,£(R m ,R n )),ve L a 2 (I,L%(Z,ir))\ (40) 

where L 2 (Z, tt) denotes the Hilbert space of R n - valued functions defined on Z which 
are square integrable with respect to the Levy measure tt. In this case the norm 
topology is given by 



m 



\vsmI= ( £ jHt)\%ndt + £ ^r(g*(t)g(t))dt + £^|^,t)||»7r(^)dA ' .(41) 



Now we are prepared to present the necessary conditions of optimality. Before we do 
so we need the following assumptions for C. 

The function C : I x R n x R n x U — > R n is measurable in t on I and continuous 
in the rest of the arguments satisfying, uniformly with respect to £ G U, the following 
assumptions 

(AQ): ^JjC(t,x,v,0\ R nn(dv)Y < K(t)(l + \x\ Rn ) 



(A7) : ^ \C(t, x, v, - C(t, y, v, < K{t){\x - y\ Rn ). 

Theorem 6.1 Consider the system (I3"5|) with the cost functional (|3"§|) and the admis- 
sible controls U ac [. Suppose {b, a, C} satisfy the assumptions (Al)-(A7) and that their 
first derivatives with respect to the state variable x 6 R n are uniformly bounded. 
An element u° G U a d, with the corresponding solution x° G 13^(1 , L 2 (Q, R n )) to be 
optimal, it is necessary that there exists a semi martingale m° G VSAi^ with the 
intensity process (if), Q, <p) G L% (I, R n ) x L 2 (I, C(R m , R n )) x L§(J, L%(Z, tt)) such that 
the following inequality and the stochastic differential equations hold: 
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(1) : E x°{t),u t - u° t ),m) + tr(Q*(t)a(t, x°(t),u t - <)) 

+ J iC(t,x°(t),v,u t -u t ),ip(t,v))ir(dv)+£(t,x (t),u t -u t )^dt > 0(42) 
for all u G U a d- 

(2) : dx°(t) = b(t,x (t),u t )dt + a(t,x (t),u° t )dW(t) + J C(t,x°(t),v,u°)q(dv x dt) 
x°(0) = x (43) 

(3) : -dtj)(t) = b* x (t, x°(t),u° t )ilj(t)dt + V Q (t)dt - Q(t)dW(t) 

+ J C*(t,x°(t),v,u°)ip(t,v)Tr(dv)dt- J ip(t,v)q(dv x dt) + £ x (t,x°(t),u°)dt 

iKT) = $ x (x°(T)) (44) 

where Vq G L§(/,iT) is given by (V (t),C) = tr(Q*(t)a x (t, x°(t), u° t ; ()), t G /. 
Remark 6.2 Define the Hamiltonian 

H : I x R n x R n x C(R m , R n ) x L%(Z, tt) x A^i(C/) — >■ i? 

by the following expression 

H(t, x, ip, Q, ip, fj,) = (b(t, x, //), ijj) + tr(Q*a(t, x, //)) 

+ / (C(t,x,v,fj,),ip(t,v)) R nn(dv) +l{t,x,n), (45) 



z 



where f G L^Zj-k). We write t — > (pit) for the L^Zjir) valued function. In terms 
of this Hamiltonian, the necessary conditions of Theorem 6.1 can be written in the 
following canonical form: 



S J H{t,x°{t),i){t),Q{t),(p{t),u t )dt>S J H{t,x°{t),il){t),Q{t),(p{t),u°)dt 

for all ueU ad , (46) 
dx°{t) = H^dt + a(t, x°(t),u°)dW(t) 

+ J Cit,x°it),v,u t )qidv x dt), x°i$) = x (47) 

dip(t) = -H x dt + Q(t)dW(t) + / ip(t,v)q(dv x dt), ip(T) = $ x (a; (T)).(48) 

Jz 

Similarly as before, one can also obtain point wise almost sure variational inequal- 
ities. 
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7 Necessary conditions with Regular Controls 

In the development of the necessary conditions of optimality given in the preceding 
two sections we have tacitly used the existence Theorem 3.2 which asserts the exis- 
tence of optimal controls from the class of relaxed controls lA a d- Let L^il x Q,U) 
denote the class of Q t adapted random processes defined on the interval / and taking 
values from the closed bounded set U C R d . This is the class of regular controls 
and we denote this by U r . It is clear that this embeds continuously into the class 
of relaxed controls through the map u 3 U r — > S u (t,u) £ U a i- Clearly, for every 
■d G L\{1 x Q, C{U)) 



£ / #(t,u,Z)Su(t,u)(dt)dt = £ / #(t,u,u(t,uj))dt. ( 49 ) 

JlxU JI 

Theorem 7.1 Consider the class of regular controls U r with U assumed to be closed 
bounded and convex. Suppose Theorem 3.2 holds for regular controls in the sense 
that an optimal control exists from the class U r . Then all the necessary conditions in- 
volving relaxed controls (Theorem 5.1, Theorem 6.1) reduce to the classical minimum 
principle for stochastic systems. 

Proof. The proof is direct. In fact it follows from straightforward application of 
the embedding mentioned above and the definition (|49l) . Considering the necessary 
conditions of optimality given by Theorem 5.1, and using the embedding mentioned 
above it is easy to derive the following necessary conditions of optimality 

(1) : £ [ {(b(t,x°(t),u t ),i;(t))+tr(Q*(t)a(t,x (t),u t ))+l(t,x (t),u t )}dt 



o 



T 



> £ / {{bit, x°(t), u° t ), m) + tr(Q*(t)a(t, x°(t), <)) + £(t, x°(t), u° t )}dt, 
Jo 

for all u G U r . (50) 



(2) : dx°(t) = b(t,x°(t),u°)dt + a(t,x (t),u t ))dW(t) 

x°(0) = x (51) 



(3) : = b* x {t, x°(t),u° t )i;(t)dt + V Q (t)dt + £ x (t, x°{t),u°)dt - Q(t)dW(t) 

i>{T) = <f> x (x°(T)) (52) 

where V Q E L a 2 (I,R n ) is given by (Vq(*),C) = tr{Q*{t)a x {t, x°{t), u°{t); ()),t e I. 

Remark 7.2 Using precisely similar arguments for the SDE with jumps, one can 
obtain the minimum principle for regular controls from those of relaxed controls given 
by Theorem 6.1. 
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8 Realizability of Relaxed Controls by Regular Controls 

We proved existence of optimal relaxed controls in Theorem 3.2 without requiring 
convexity of the control domain U. In any application it is much easier to construct 
regular controls. So one may be interested to find a regular control corresponding 
to which the performance of the system is close to that realized by optimal relaxed 
control. In this regard we have the following result. 

Theorem 8.1 Consider the regular controls U r with U closed bounded but not 
necessarily convex as in Theorem 7.1. Suppose the basic assumptions of Lemma 3.1 
and Theorem 3.2 hold and consider the control problem as stated in Theorem 3.2. 
Further, suppose that x — > $(a;) is continuous. Let u° G U a d be the optimal relaxed 
control. Then, for every e > there exists a regular control u r G W such that 

J(Ur) <6 + J{u°). 

Proof Since U ad = L^(J x Q, M\{U)) C L^(I x Q,M(U)) is compact in the vague 
topology (that is weak star topology) and convex (because M.\{U) is convex, it follows 
from the well known Krein-Millman theorem that 

U ad = cl v conv(ext(U ad )), 

that is, U a d is the weak star closed convex hull of its extreme points. Considering the 
embedding W > U aa as mentioned above, it is easy to verify that the extreme points 
of U a d are precisely the set of regular controls W through the map u 3 U r — > 5 U G 
U a d- Thus, if u° G U a d is the optimal (relaxed) control there exists a sequence {u n } of 
the form 

n n 
i=l i=l 

such that u n — > u°. Let {x n , x°} C B^(I, L 2 (0, R n )) denote the solutions of the sys- 
tem equation (TjQ) corresponding to the controls {u n ,u°} respectively. Then it follows 
from Lemma 3.1 that, along a subsequence if necessary, x 11 x° in -6^(7, L 2 (f2, R 11 ))- 
Consequently, it follows from continuity of I and $ in the state variable x and the as- 
sumptions (al)-(a3) and Lebesgue dominated convergence theorem that lim^oo J{u n ) = 
J(u°). Note that for every n G N, u n G W, and so, for every e > 0, there exists an 
n e G N such that \J(u n ) — J(u°)\ < e for all n > n £ . Taking u r = «" E we have 
J(u r ) < e + J(u°). This completes the proof. • 

Remark 8.2 In view of the above result it is evident that an e-optimal control can 
be found from the class of regular controls (measurable functions with values in U) 
though the limit of such controls may be a relaxed control. More specifically if U C R 
consists of a finite set of points, it is clearly non-convex, and optimal control may 
not exist from the class of regular controls U r based on the set U. However, optimal 
relaxed controls do exist. In this case the sequence of regular controls approximating 
the optimal relaxed control may oscillate violently between the finite set of points of 
U with increasing frequency (converging to infinity). This is known as chattering. 
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